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Abstract 

Effective and efficient representation of color features of multiple video frames or pictures is an 
important yet challenging task for visual information management systems. Key frame-based methods to 
represent the color features of a group of frames (GoF) are highly dependent on the selection criterion of 
the representative frame(s), and may lead to unreliable results. We present various histogram-based 
color descriptors to reliably capture and represent the color properties of multiple images or a GoF. One 
family of such descriptors, called alpha-trimmed average histograms, combine individual frame or image 
histograms using a specific filtering operation to generate robust color histograms that can eliminate the 
adverse effects of brightness/color variations, occlusion, and edit effects on the color representation. We 
show the efficacy of the alpha-trimmed average histograms for video segment retrieval applications, and 
illustrate how they consistently outperform key frame-based methods. Another color histogram 
descriptor that we introduce, called the intersection histogram, reflects the number of pixels of a given 
color that is common to all the frames in the GoF. We employ the intersection histogram to develop a fast 
and efficient algorithm for identification of the video segment to which a query frame belongs. The 
proposed color histogram descriptors have been included in the ISO standard MPEG-7 after extensive 
evaluation experiments. 
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